Skip to main content

Tunnel deployment instructions

Prepare id and certificate​

  1. Generate a new unique id for the customer's tunnel by running the following in Python:

    import uuid
    print(uuid.uuid4().hex)

  2. Download the tunnelproxy-ca.pem file from the AWS secrets manager il-central-1 region from either dev or production AWS account

The following steps will require the generated tunnel id and downloaded tunnelproxy-ca.pem file so make sure to keep them until the deployment is completed

Create a new Cloudflare tunnel for the customer​

  1. Go to Cloudflare tunnels dashboard
  2. Click 'Create a tunnel'
  3. Choose type 'Cloudflared'
  4. Enter the tunnel id as the tunnel name and save
  5. Copy the tunnel access token after the tunnel is created
  6. Click next and under 'Published applications' tab add the following:
    • Subdomain - the tunnel id
    • Domain - legion-tunnel.com
    • Service type - http
    • Service URL - localhost:8080
  7. Click 'Complete setup' to finish creating the tunnel

The following steps will require the tunnel access token so make sure to keep it until the deployment is completed

If adding the first tunnel in a new region - update Cloudflare source IPs list

  1. Go to the Backend public IPs Zero Access list in Cloudflare
  2. Add the 2 public IP addresses of our backend in the relevant AWS region, or the single public IP address of the GCP region, to the list and save the list to allow those IPs to call the tunnel
  3. Go to the Backend public IPs security allowlist in Cloudflare
  4. Add the 2 public IP addresses of our backend in the relevant AWS region, or the single public IP address of the GCP region, to the list and save the list to exclude those IPs from WAF rules (to avoid dropped calls if the customer data we send matches on some rule)

Create dedicated customer proxy instance​

  1. In infra repository's __main__.py server infra file find the list of proxy_instances for the customer's region and add a proxy instance for the customer. The instance name should be the Legion customer id (org_...).

  2. Merge the change and deploy the infra

  3. Open the instance in the relevant cloud portal and note down its internal host value (will be used later as proxy URL host):

    • AWS: in AWS console go to EC2 -> Instances (under production account and customer's region), select the tunnel proxy instance by name, and copy its private DNS name
    • GCP: in Google Cloud console go to Compute Engine -> VM instances (under the project for the customer's region), select the tunnel proxy VM by name, and copy its internal DNS name or internal IP from the instance details

  4. Connect to the instance:

    • AWS: in EC2 click 'Connect' -> 'Session Manager' -> 'Connect'
    • GCP: in Compute Engine -> VM instances click the SSH dropdown on the VM row and choose 'Open in browser window'

  5. (Optional but recommended) run bash in the opened terminal to use a bash shell. For consistency across engineers, run all setup commands below with sudo.

  6. Create a folder for the proxy's files under /usr/proxy by running:

    sudo mkdir -p /usr/proxy
    cd /usr/proxy

  7. Copy the following files to the /usr/proxy folder:

    • The mitm_script.py, mitmproxy.service and .env files from the /tunnel_proxy folder in this repository. Make sure to update the values of the .env files (tunnel DNS host taken from tunnel creation step - should be <tunnel-id>.legion-tunnel.com, client id and secret taken from AWS secret manager il-central-1 region under 'infra_secrets' as TUNNEL_PROXY_CLIENT_ID and TUNNEL_PROXY_CLIENT_SECRET)
    • The tunnelproxy-ca.pem file (download from AWS secrets manager in the preparation step)

    Verify that all files were copied by ensuring ls -a /usr/proxy shows 4 files.

    GCP only: before continuing, update mitmproxy.service:

    • Replace /home/ssm-user/.local/bin/mitmdump with /usr/local/bin/mitmdump
    • Replace User=ssm-user with User=root
    • On AWS keep existing ExecStart and User=ssm-user unchanged

    (The fastest way to copy the files is to copy them as text - open them in a text editor on your machine, copy all text, run sudo nano <filename> in the EC2 instance, right click, paste, save)

  8. Prepare the expected folder structure and permissions:

    sudo mkdir /usr/proxy/certs
    sudo mv /usr/proxy/tunnelproxy-ca.pem /usr/proxy/certs/mitmproxy-ca.pem

    Set /usr/proxy to root-owned so any engineer with sudo access can manage it: (mainly needed for GCP which doesn't use a shared user)

    sudo chown -R root:root /usr/proxy

  9. Install mitmproxy:

    AWS:

    sudo apt update
    sudo apt install python3-pip libffi-dev libssl-dev -y
    sudo pip3 install --upgrade pip --break-system-packages
    sudo pip3 install --no-cache-dir mitmproxy==11.0.2 --break-system-packages

    GCP:

    sudo apt update
    sudo apt install python3-pip pipx libffi-dev libssl-dev -y
    sudo pipx install mitmproxy==11.0.2
    sudo ln -sf /root/.local/bin/mitmdump /usr/local/bin/mitmdump

  10. Configure mitmproxy to run automatically on system boot (in case the VM will restart)

    sudo mv mitmproxy.service /etc/systemd/system/mitmproxy.service
    sudo systemctl daemon-reexec
    sudo systemctl daemon-reload
    sudo systemctl enable mitmproxy
    sudo systemctl start mitmproxy

  11. Verify everything is configured correctly by checking service status and ensuring it is running successfully by checking that service status is 'active (running)' written in green. If there's an issue, the errors should be shown in the command output

    sudo systemctl status mitmproxy

If service is showing errors or failed to load, there are a few ways to debug the issue:

  1. Copy the exec command from mitmproxy.service file and run it directly from terminal. (Note: this execution method doesn't load environment variables, so ignored errors related to them)
  2. View and tail service logs by running sudo journalctl -u mitmproxy.service -f

After making any changes run sudo systemctl restart mitmproxy to restart the service and run the status command again to see if the issue was resolved.

The following steps will require the proxy instance internal host value (AWS private DNS, or GCP internal DNS/internal IP), so make sure to keep it until the deployment is completed

Create proxy mapping for the customer​

  1. Open the proxies mgmt page for the customer's region (e.g https://usw2.legionsecurity.ai/mgmt/proxy)
  2. Add (or update if it already exists) the tunnel mapping section for the customer to map the tunnel id to proxy instance URL. Make sure to set the proxy URL as http://<proxy-instance-internal-host>:1380 (for example AWS: http://ip-172-31-28-15.il-central-1.compute.internal:1380, GCP: http://10.10.0.25:1380).

For example: { "08eb9c560fe34319bbb0d9bfe0a0680f": "http://ip-172-31-29-161.us-west-2.compute.internal:1380" }

Prepare files to provide the customer​

  1. Prepare the following folder structure to provide the customer:

    • config folder containing:

      • dlp_config.json file. Should be exported via the org settings mgmt page for the customer's region (e.g https://usw2.legionsecurity.ai/mgmt/settings). Example output:

        [
           {
              "mask": "Clinical Trial",
              "regex": "\\b(NCT|nct)\\d{8}\\b",
              "flags": ["g"]
           }
        ]

        Important: for the following rules, regexes must use (?:^|\\s)\\b as prefix and \\b as suffix:

        • Credit Card Number
        • US Fax Number
        • US Phone Number
        • Med License
        • Med Record

        Otherwise they can match random strings in JS files returned through the tunnel and break flows.

    • .env file with: (fill in the tunnel access token from tunnel creation step)

      TUNNEL_TOKEN=<tunnel-access-token>
      DLP_CONFIG_FILE_PATH=/config/dlp_config.json

      Optionally, if customer has internal servers we need to call with HTTPs that use certificate from custom CA, the customer can create a folder with the cer/crt files of the custom CA and add the following to the .env file: (any path is ok as long as it's volume mounted to the container)

      EXTERNAL_TRUSTED_CERTS_DIR=/config/custom_ca

      Optionally, if customer upstream requires mutual TLS (client certificate authentication), set the following environment variable and mount the referenced folder to include per-profile certificate/key files:

      CLIENT_CERTS_DIR=/config/client_certs

      Expected structure example:

      config/client_certs/<profile-name>/cert.crt
      config/client_certs/<profile-name>/key.pem

    • Tunnel Docker credentials.txt file containing the username and password for the ACR image repository containing the tunnel image. The credentials can be found in 1Password under 'LegionTunnelAcrCustomerPull'.

  2. Zip the folder, set its name to be {org_id}_{tunnel_id} (e.g org_prPpKwsevsceYv04_tester.zip) and upload it to the legion-tunnel-files storage bucket on AWS/GCP in the customer's region. Once the file is uploaded the customer would be able to download the file and check tunnel run status from the webapp (https://app.legionsecurity.ai/settings/tunnel)

Tunnel execution in customer side​

Note: this part of the instructions is customer facing and is also available in our documentation at https://docs.legionsecurity.ai/administration/tunnel#installation

  1. The tunnel should be run by executing the following in the unzipped folder:

    sudo docker login legiontunnel.azurecr.io --username <LegionTunnelAcrCustomerPull-application id> --password <LegionTunnelAcrCustomerPull-application secret>

    sudo docker run -d \
       --env-file .env \
       -v $(pwd)/config:/config \
       --name legion-tunnel \
       --pull always \
       --restart unless-stopped \
       legiontunnel.azurecr.io/legion-tunnel:latest

    Notes:

  2. After customer successfully ran the docker container verify the container is running correctly by:

    • Having the customer run sudo docker ps -a to verify the container started correctly and is running
    • Having the customer run sudo docker logs <container id> (with the container id from the docker ps command) to verify that are no error logs from the container
    • Checking the tunnel status in Cloudflare tunnels dashboard - it should be marked with a green 'Healthy' state

    These checks will verify the tunnel is connected and the container is running, but we have no way to verify the DLP proxy until trying to run an automation through it

If there are issues with the tunnel we can ask customer to add VERBOSITY=debug flag to the .env file and restart the container, so that docker logs will show much more verbose data for us to debug the issues.

Enable proxy for autonomous investigations​

  1. Open the proxies mgmt page for the customer's region (e.g https://usw2.legionsecurity.ai/mgmt/proxy)
  2. Update the existing proxy mapping for the customer (that we created previously with the tunnel mapping) and add mapping for relevant tools/domains that should have their traffic routed through the tunnel. Make sure to set the proxy URL as http://<proxy-instance-internal-host>:1380 (for example AWS: http://ip-172-31-28-15.il-central-1.compute.internal:1380, GCP: http://10.10.0.25:1380) and the proxy type to 'tunnel'
  3. You can use the health check and telemetry pull options to verify the connection to the tunnel running on customer side is working correctly and debug issues if autonomous traffic isn't working correctly. For more debug steps see the TSG - Tunnel troubleshooting